With the advent of digital photography, consumers are amassing large media collections of digital images and videos. For purposes of this description, the term “images” will be understood to include both still images and “videos” which are a collection of image frames, often having associated audio stream. Therefore, an image collection can contain images or videos or both. The average number of images captures with digital cameras per photographer is still increasing each year. As a consequence, the organization and retrieval of images and videos is already a problem for the typical consumer. Currently, the length of time spanned by a typical consumer's digital image collection is only a few years. The organization and retrieval problem will continue to grow as the length of time spanned by the average digital image and video collection increases.
The automatic organization of a media collection, either as an end in itself or for use in other applications—has been the subject of recent research. Relatively sophisticated image content analysis techniques have been used for image indexing and organization. For image indexing and retrieval applications, simple text analysis techniques have also been used on text or spoken annotations associated with individual images or videos. The recent research has involved a number of techniques and tools for automatic albuming of images and videos.
Date and time information from the camera has been used to perform event segmentation, as for example described in U.S. Pat. No. 6,606,411 by Loui and Pavie. An event consists of a set of images of videos related to a common event, for example “trip to the beach.”
U.S. Pat. No. 6,810,146 by Loui and Stent, described extracting certain types of information from spoken annotations, or the transcriptions of spoken annotations, associated with photographs, and then using the results to perform event segmentation, identification, and summarization.
Certain applications allow for viewing images on a timeline. In essence, the images can be viewed or sorted into consecutive order based on the image capture time. For example, the application Picasa 2, distributed by Google, has a timeline view where groups or sets of images and videos are shown to the user and each image set has an associated time (e.g. “April 2005”). In addition, the Adobe application Album 2.0 has a calendar view where calendar pages are shown and small versions of images captured on a specific calendar date are shown on that date. In each case, the software groups images related by capture time into sets. However, the sets of images are not labeled with meaningful names other than the capture date or date range. Thus, the only calendar information used by these applications is the day, month, and year. They do not use any occasion (e.g. Thanksgiving) or appointment (e.g. vacation trip to Florida) information to label the images with meaningful names.
Furthermore, U.S. Pat. No. 6,108,640 describes a method for determining periodic occasions such as holidays. However, there is no description of assigning meaningful labels to images or sets of images.
In UK Patent Application GB2403304A, Rowe describes a method of labeling images with labels based on the image capture dates corresponding to national events for later use in text-based search and retrieval of images. This method cannot provide for the fact that for many people, many national holidays are not observed. For example, few people actually celebrate Groundhog Day. Subsequent searches by a user for images of “groundhogs” would return images captured on February 2. Furthermore, since may consumer images are taken on occasions that are not associated with national holidays (such as the birthdays of family members, or vacation trips), this method cannot provide useful labels for most consumer photos.
In U.S. Patent Application Publication U.S. 20040201740A1, Nakamura and Gibson describe a method of placing images into storage locations based on calendar information. Their method does not provide for automatic annotation of images.
In U.S. Patent application Publication U.S. 20050044066 A1, Hooper and Mao describe a calendar-based image asset organization method. Their method allows people to indicate via a graphical user interface a date range of interest. Images captured during that date range are then retrieved for the user. The method does not provide for automatic annotation of images.